A comparison of generalized linear discriminant analysis algorithms
نویسندگان
چکیده
Linear Discriminant Analysis (LDA) is a dimension reduction method which finds an optimal linear transformation that maximizes the class separability. However, in undersampled problems where the number of data samples is smaller than the dimension of data space, it is difficult to apply the LDA due to the singularity of scatter matrices caused by high dimensionality. In order to make the LDA applicable, several generalizations of the LDA have been proposed recently. In this paper, we present theoretical and algorithmic relationships among several generalized LDA algorithms and compare their computational complexities and performances in text classification and face recognition. Towards a practical dimension reduction method for high dimensional data, an efficient algorithm is proposed, Preprint submitted to Elsevier Science 27 June 2007 which reduces the computational complexity greatly while achieving competitive prediction accuracies. We also present nonlinear extensions of these LDA algorithms based on kernel methods. It is shown that a generalized eigenvalue problem can be formulated in the kernel-based feature space, and generalized LDA algorithms are applied to solve the generalized eigenvalue problem, resulting in nonlinear discriminant analysis. Performances of these linear and nonlinear discriminant analysis algorithms are compared extensively.
منابع مشابه
Comparison of statistical pattern - recognition algorithms for hybrid processing . II . Eigenvector - based algorithm
The pattern-recognition algorithms based on eigenvector analysis (group 2) are theoretically and experimentally compared. Group 2 consists of Foley-Sammon (F-S) transform, Hotelling trace criterion (HTC), FukunagaKoontz (F-K) transform, linear discriminant function (LDF), and generalized matched filter (GMF) algorithms. It is shown that all eigenvector-based algorithms can be represented in a g...
متن کاملWeighted Generalized Kernel Discriminant Analysis Using Fuzzy Memberships
Linear discriminant analysis (LDA) is a classical approach for dimensionality reduction. However, LDA has limitations in that one of the scatter matrices is required to be nonsingular and the nonlinearly clustered structure is not easily captured. In order to overcome these problems, in this paper, we present several generalizations of kernel fuzzy discriminant analysis (KFDA) which include KFD...
متن کاملUncorrelated trace ratio linear discriminant analysis for undersampled problems
For linear discriminant analysis (LDA), the ratio trace and trace ratio are two basic criteria generalized from the classical Fisher criterion function, while the orthogonal and uncorrelated constraints are two common conditions imposed on the optimal linear transformation. The ratio trace criterion with both the orthogonal and uncorrelated constraints have been extensively studied in the liter...
متن کاملCharacterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems
A generalized discriminant analysis based on a new optimization criterion is presented. The criterion extends the optimization criteria of the classical Linear Discriminant Analysis (LDA) when the scatter matrices are singular. An efficient algorithm for the new optimization problem is presented. The solutions to the proposed criterion form a family of algorithms for generalized LDA, which can ...
متن کاملNonlinear Discriminant Analysis Using Kernel Functions and the Generalized Singular Value Decomposition
Linear Discriminant Analysis (LDA) has been widely used for linear dimension reduction. However, LDA has some limitations that one of the scatter matrices is required to be nonsingular and the nonlinearly clustered structure is not easily captured. In order to overcome the problems caused by the singularity of the scatter matrices, a generalization of LDA based on the generalized singular value...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Pattern Recognition
دوره 41 شماره
صفحات -
تاریخ انتشار 2008